associated for individuals who do not finish high school. we have heard a little bit about international comparison this morning. when you look at how the u.s. is performing on different international test, we are well behind. we don't do well, but we are way behind in math and reading and science at various grade levels. so how do we overcome this problem? lots of important ideas. we heard earlier that there is a system approach. but what is motivating our proposal is that we take this american strength, which is innovation, we continue to lead on this. how do we take that creative engine and get it to k-12 education so we we can see improvements and qualities and outcomes and achievements for

our students? >> if you look through history, whether it is transportation, our children listed in age that our ancestors could only dream of. we have computers, we have all sorts of new things. that really doesn't seem to happen until k-12 education. if you look at research and development, in k-12 education, it is one 15th the rate, of the u.s. economy overall. it is 152-1100 rate of what we see in innovative sectors like health care. what is getting in the way? well, what we try to do is first diagnosed what is giving away. i wonder standing back, we see what can we do about it.

and that is essentially a layer in which we hope to create a creative marketplace that succeeds in what normally seems to be stagnant. we have k-12 schools, 14,000 school systems. we have would-be innovators and entrepreneurs. if something seems to be separating us. what is going on in the middle? this is underlined a lot of other concerns, but what is the problem is that people don't really know what works. some people might know what works, but even if they know personally, they have trouble convincing others. with education technology, we have no particular way to evaluate claims. we have a lot of abilities to assess how we try to teach the particular goals of the systems and how it ends up. it ends up to huge barriers for

the entrepreneur. they can come up with something that i think might be a great tool in the classroom. but we have to convince everybody that it will actually work. they have to go school system by school system and they engaged arduously to convince people to use their product. and that is only one school system for entrepreneurs. entrepreneurs don't survive very long if they can't find anyone to help out. there are huge barriers. if you look at healthy innovations, take a smart phones, many of you have smart phones now, tablets, the internet, then apple comes up with the iphone and the operating system and android is the computing platform. within two years, we have 85 new different companies -- excuse me some 85,000 new different companies developing applications. why is that not happening in education? it's just not worth it.

so how do we make it worth it? we try to make it worth it. we have to focus. we have to focus on the kind of education technology and some is very hard to do. ..

what is going to do if it's going to be in evaluating and reporting platform, like a consumer report that will tell you these things work. but it's also rigorous evaluation. because whoever the québec charter schools for using a lottery system talking about before. the idea here is another model comes in many every day when you go online. other people see how it affects them. the internet allows you to have real-time randomized control. digital learning tools allow some good seasons and kids not been then report the results. we know if a particular software really is good at achieving that goal among the state standards. i put to death to let you know we've talked about this carefully. it is simple we take a relatively low cost to do it

cannot free, but if we can lower entry barriers will open up the opportunity for innovators to rationing. ultimately it's a community proposition . we can do this for $5 million. it's a pretty simple technology used by corporations all the time in consumer reports act in online and will disseminate findings to teachers. there's a lot of partners and content. one example is the last additionally deliver content, technical support, week of innovative schools is a new collection of school and school systems dedicated to his innovation. think about the university hospitals for each evaluation in every hospital gets to use them. a small set of schools is going to become part of a testbed to evaluate promising technology. catalyst is about tools for teachers. we think teachers would be a huge part, both in terms of the

tools he is in the classroom, the source of great ideas in the same resurgence are the source of medical device innovation and teachers to fan out this works. we can deliver that. we think and we hope it can make a huge difference. thanks. >> thank you. >> okay, another really distinguished group of people here and will introduce them as they go down the row asking questions. i think we'll start to my last with eric westendorf, former teacher, former principal of successful charter school here in washington d.c. and is the cofounder and ceo of learned zillion, which is a learning platform for most software. so as someone who has created a project that is in principle interact team with edu star, i

wonder if he be khatami which you think about this wacky economic idea. >> absolutely. a little bit of background about where learned so and started from. as the principal at a charter school in washington d.c. and during that time is principle, we were seen some really good gains in terms of our student learning the nature but is that two things. one was outstanding staff that the staff talked about democracy and we just recruited amazing teachers who work incredibly hard. in fact, one is in the second row here, amy holmes is her fourth-grade literacy teacher when i was there. and the second is who paid attention to data. so we were really deliberate about looking at where students ranks where and where their weaknesses were and then being thoughtful about how we would address the weaknesses. it wasn't not affect at the idea for learnzillion emerge because

i was frustrated that i wake up better and better at analyzing, did not say we were getting as good at turning this insights into very targeted actions. both in terms of teachers and in terms of parents. in terms of teachers, the teachers still had a curriculum ahead of them that they needed to get to and there was one of them in 25 students present giving the targeted differentiated instruction as needed was really difficult. i should also say two of my kids go there and my daughter who is on the second grade would come home and be working on her mass. and here i was as a parent wanting to help her. i was there principle. i hired her teacher and have been involved in the decision around the curriculum. i would sit there feeling like i don't know how to help her in a targeted way. like this is absurd that i am in this unbelievably privileged

position in nature is one of the most important people in my life doing the most important thing is i've come to get an education and it's the one area or as a parent i do not have any transparency, even though i'm her principle and i hired her teacher. how do we provide a transparency anyway that can target instruction to the needs of students? so out of the question i started talking with teachers and we said what is -- what if we were able to capture expertise or teachers have in a way that could be accessed anytime, anywhere by students, teachers and also parents. so we started to create these very short videos based on, corestates tanners, much like tapping into expertise of great teachers and then it grew from there, were restarted to say what if it wasn't just amazing teachers, but teachers from

charter, private schools around the country we did this in an organized fashion. so that's what we're doing right now. it's interesting reading the report that my first reaction was this is exactly what adam frankel at digital promise and i have been talking about in terms of new technology. how do we rapidly test it, right-click suddenly see a paper and call them up. of course ciardi knew them and has been talking to them. so i think this is really what because entrepreneurs starting something new -- in many ways i was in a lucky position because i was in the schoolcommissars it's getting a prototype and test really quickly within several weeks time of what made sense and what didn't makes sense. and so that was something to the extent that we can make that more systemic so much printer is a don't have to be principles and safe schools can the feedback. i think that is critical. i will say one of the challenges

is from sort of my side, looking at the prospect of the result made public. on the one hand they like it because they would be very strong. at the same time can i also am a big proponent of building started movement in the main concept kind that is you are not going to have the right answer when he starred as much per number. you basically take a leap, have an initial idea and then you are practically learning and figuring out of the things you do wrong to try to create product markets that save. so i'm either into learnzillion i feel it would do that every week, testing things, seeing what teachers like him at going into school and talking to about it. it is a little scary to think that given that process, suddenly we would be given a score that all teachers, districts and so forth would say that's for learnzillion

strength, three star when i know will be for a start and we can ratchet up overdoing. that's one part that makes me nervous. the overall idea of creating infrastructure were good ideas can be tested and folks can make really thoughtful strategic decisions are a targeted action is really exciting. >> thank you peered we're fortunate to have dennis van roekel stay around. a lot of come and go common coeducational run through the teachers at the end of the day. so there was any mascot of the metric system. it was the 1980 version. i wonder if you could talk a little about how teachers today see at tackiness of a complement to what they're doing quite >> good question. i remember when we are fighting for the rate so we could get internet in the school is inaccurate and sitting with

people -- by the way, i am an immigrant when it comes to technology. my grandchildren are natives. it is such a learning curve for all of us in a profession that is being dramatically impacted by technology. a couple things. first obviously we support this in one of the things i think about is making sure we're asking the right questions. years ago i'm old enough that i remember when we were debating about calculators in the classroom. there was this huge debate of whether students should be allowed to use calculators. people fought on both sides. there was a yes or no, but we should have been asking, when is it appropriate to use a calculator and when isn't it? i think that is part of the tech elegy. it is not yes or no. it is, when is it appropriate to accomplish what purpose? so asking the right question as we go through is really important in one of the debates going on right now is all or

nothing, totally virtual schools versus the blended. we are much more in the sight of supplementing rather than simply an team. we believe there has to be something in between and so we see that. another area that is very relevant is the whole issue of equity. i'm the one hand, we see technology as a huge positive tool for leveling the playing field. if you're depending on students to have a level playing field, it will never happen. so how do we provide through education a place where all students really do have the same shot so that i'm not on the dialogue and you're on broadband. it is such an unfair disadvantage to a student. so when people say what will schools look like in 20 years? i don't know, but it's hard for me to imagine that there will always be a place in the

neighborhood were all of us have the same tools as a student, the same opportunity to pursue agreement and same access. on the edu star, one of the challenges is the incredible push for someone to own that because it's a very powerful thing if you own edu star and you decide who is a good idea and a bad idea. it's hard to resist that. another challenge that i see in teachers are so excited about use of technology. your example about the apple computers, i can't tell you as computers when the schools, how many schools you can walk into a see a storage room full of computers that were never said that. because in this system, someone figured out how to find the capital outlay money to buy them. but in my system, no one ever said gosh, do we have to train anyone on how jesus in the

classroom? i was just in an elementary school that has a smart burden of a classroom. now if they brought this into my new classroom, where will i learn how to use it effectively as a tool to teach children? i don't learn that by osmosis. when i unpacked the box, it didn't come to me. and in my system, unemployed by district for a certain number of days, like 185 and 180 of those are in meetings all day long. i am teaching. so when is it that i've learned this this new technology? and upon all of us become gray hairs to leave because they weren't there at the beginning. i think we have to think about building into the system the time or training, not on your own time, not evenings and weekends, the part of my job to kn how to use this. the last thing i would say --

i'm going to stop there. >> thank you. karen cator is the director bought this educational to elegy for department of education. she previously worked at apple. you can direct all of your compliments or complaints about apple products to her afterwards. she's going to stand on the side. karen, i wonder if he could talk about the federal role in all of this. >> one of my backgrounds inside public education, so that's actually in terms of the lives of seeing how this can fully impact the opportunity to learn. you know, thank you for sticking with this. they been on this topic for quite a while focusing on what are the barriers and comparing with other marketplaces in trying to come up with one accelerant that i think it's actually really interesting.

the other thing i really think is articulating i went to restate because it is an important notion and gets to dennis' point. a lot of times people say things like having a technology works? should be his technology in education? or how much screen time is enough screen time. the answer to the question is it depends. depends what's on the sunscreen, what's, what's going on, what's the interaction? what's the point? is a for learning, entertainment, so you can have a quiet dinner. what they did is laser focused on instructional software or instructional tools. the second thing i want to say is a few things have been in the marketplace to make this very different the 1980s version of computers and a classic, not the least of which is a tremendous advancement into elegy. so someone recently said to me

our teachers don't know how to use ipods. i said really? like really don't know how to turn it on? bearlike yeah. so we have the next that is hung with us. a lot of the technologies are easy to use and they don't require what to click on, how to turn it on. however, the point is incredibly important. they require a lot of rethinking about the kinds of things that students and teachers can do in the learning context when everybody is powered up with the internet, with primary source documents, explanations of complex comments from the teachers had identified some from multiple other entities. they are powered up at the animations and simulations and models of complex scientific contact and accessing experts that can help them understand

the buzzes in their backyard or how hurricanes happen or whatever it maybe they are interested in. so this will opportunity to learn right now is incredibly powerful if we figure out how to harness technology and think about it within the context of our current system of public education. absolute important point about equity. we can't just relegate this to i don't know. we don't have the digital divide, whatever. public education is the place we need to make sure that everybody is fully connected and powered up so that they can participate in this opportunity to learn. one of the emerging gaps is the gap between people who use technology for personal empowerment to apply for a job, find information, get health information or services on the lake to learn as people use it for personal entertainment and

maybe light communication. education is the place we can fully teach and work with everybody. teachers, students, parents cannot understand how to leverage powerful tools for teaching and learning. so in terms of edu star, what i like about it as they focus on instructional technology. the second is a push is on something incredibly important that we don't have an education right now and that is the notion of smart domain. we don't have a whole population of people, professional educators who have had taught how to we think about the kind of materials they use because in general, adapted materials were shipped into the classroom and that is that they started with. so we are at a point now where we can benefit from a much smaller ratcheted up demand-side and the kinds of information they propose that being a system like edu star is incredibly helpful. you can do x or p. ration and

then you can do this kind of kitchen environments where you can fully test things. there's lots of thorns around the edges. it's not as simple as that. there's implementation, lots of other things, not the least of which is that you pointed out that none of these tools are static. we are used to having things you can hope to build long enough to fully test them and that is not the situation with online emerging digital tech elegy and i'm ironman. if we create a system with the data produced is also feeding back to create smart supply, to feedback vendors and developers have the best information about what is happening in classrooms by observing, looking at data from the sea was happening, talking users come and see they need help navigating to create a

much more powerful and intelligent supply. so all of this is really important in the bottom line is it's important because previous panels put it out. ready to fully invest and improve the opportunity for every american from adults undereducated in these new jobs to young children who are not having the same language as their neighbors perhaps. >> thank you for a match. our next panelist is bill tucker for u.s. policy not to see at the bill and melinda gates foundation. we're fortunate to have him here with us and i thought i would ask you, what technologies do you see out there? and how might they interact with edu star that we are kind of at the cutting edge here. >> first let me say thank you for the paper because this is incredibly important subject

because no matter what the cool new tech elegies or ideas that are out there, if we don't have a way to make sure they are actually helping students and helping teachers and they don't actually make a into not just one, but many, many classrooms, then it doesn't matter. so finding a way to help break that sort of failure that she talked about is really critical. i think there's a couple different kinds. clearly there's a lot of entries and intelligent tutors that can interact with students and help provide feedback and also i think the most powerful ones will work really closely with teachers, not replace teachers, but the benefit is really help teachers help them with access to data, help and use information and help them make

their decisions and that is what we see in other professions that the autopilot does it replace the pilot. the autopilot helps the pilot helps appellate decisions and take some of the burden off of the pilot. those things will be very powerful. i'm really excited about a lot of the innovation about video and being able to see these types of process. so both on the students died, wow, i can actually review and learn this lesson in different ways to six years in a different way. it is interesting to have the opportunity to learn more about what eric was doing a year and half or two years ago and i talked with one of your the teachers, who is one of your best teachers trying this out. she had this whole other the perspective she said, first of all, to these videos helps you think about her is and how to really condense it down.

but what i thought was even meaner because teaching is so difficult i don't expect everyone is make videos and other spare time, but we really talked about providing directors cuts. so what is one thing -- it's very difficult to get feedback or so to be able to watch someone whose she's great at their craft and hear them talk about what they're doing and why they do it is a really neat capability and something that has a lot of promise. so that's a lot of promise for helping with construction. the final one i will mention the sub dean directly related to what you're doing in trying to solve a similar problem and this is something that the gates foundation has invested than ever starting to roll out in several states and districts, still pretty early. it's one of those things that is the underground, basically the plumbing to put all this

together because a lot of what they talk about is part of the promise here is these tools, the rapid testing, you have a lot of data and you can use the data to help make your decisions are really other things, not just all or nothing, but it was helpful for the students who started here or in this context so it's not just that judgment you are talking, eric, but we get information so helps us understand which students and cases of how can this be better but complementary to something else were doing. some of things to work on is the shared learning initiative and this is the way to sort of take all of these bits of information that are stored inside a. so maybe if you're a teacher you stop on instructional programs for the set of students, for this piece on this for this piece on this for this piece and all of that is in separate

places so you never see all those things together in one place. it is like it is your doctor had xers over here in a test over here you couldn't put them together in the right way to get the whole picture of the patient. but it's kind of the way we are right now to do that is incredibly expensive so that leads to problems you talk about in your paper, were let's not worry about it and not cesspit let's get the sense that this together and help them with their current plumbing data system. that is one of the things, but has a lot of potential to open up better avenues for educators and districts to make your decisions and help get evidenced in the way. in some ways that type of tool would make what they try to do a lot easier. >> thank you. i think you're the designated responder here. >> first of all, thank you for

putting together this great group of panelists. this is fantastic here there's so many great points are for paper. afterwards i budget talk to each of you because it's all going to make the proposal better and stronger. one note for the younger folks in the audience, maybe 30 or should not go on a panel like this and somebody will ask you, what was life like before the app store? they'll be shocked that because you're old, but because they can't imagine life existed before the app store. what you see now is so incredible that what made the market, where these entrepreneurs come from. what it is his but which are getting a lower barriers to entry. that's why you can download things on your tablet or smartphone. but we want to do is take over the edge for normal energy and innovation coming out of the u.s., but around the world and use it to give teachers better schools in the classroom. it's consumer reports for educational technology, the

innovation energy around the world to develop their tools for teachers and complement the work they do in the classroom. the panelists got it right in terms of what we do. a few points worth thinking about hard as first the issue about power, who controls edu star? the person who controls the platform will have enormous power and the need to be careful. our proposal now is to set this up as a five o. one c. three piece were not going to allow anyone to edu star to make investments in educational technology companies. consumer reports is a great outline. they don't take examples. we will make sure that were not bypassing results. if you. if you got the whole system falls apart, so that is something we think a lot about. the second thing karen brought up in dennis is about equity. so it's really important issue you can do that to elegy and

issues around the digital divide. when they were happy with a edu star is the ability to pick up heterogeneous effects. lots of great technology might work for one it's not the other. the wonderful thing is your connection to pick that up and figure out what works for hk. personalized learning is analogous to the revolution health care. some equity issues can mitigate and that's another thing so be a big priority for the platform. two other points responding to these. this is one of the most exciting trends right now come people think you are it to quick iteration. right now you set up a product, figure out what works and go back to the drawing board. what may be a shame if you can't do this at the start platform, prodded by brady and you could come back and do it again. so we also the system for prototyping so entrepreneurs can come and try a version of the program, see out to, go back and improve it and get up on the

platform a month later. the last thing, which i really like that was said by a couple panelists asserted translating data into action. you saw on the first two panels how many smart people we have in academia, school level, government, generating data about education. the question is how do you translate that data into feasible action? that's where we think edu star can make sure the data just doesn't pay our computers or hard drives. it gets back to peoples that they can make decisions, parents and teachers. there's a lot of other good points, but rather hear from the panelists on the q&a. but i really appreciate all the comments. >> one thing i wanted to add, too was one of the most positive things when i read your paper and what you're doing is for the part tichenor, it's an extremely important need and right now there is so many innovators and

ideas out there that they are everywhere and there's got to be a way to sort through the period so i think that is part of the edu star concept is so, so important. it is like right now in most places people talk about the common core standards and those are done. they've been adopted by 46 states. now is the implementation and it is a huge, important stat. and what will we be doing differently in classrooms? almost everything you see now says aligned with the common core standards. i don't believe that. but they say it because it's a good marketing mind to go to your new software curriculum. part of the concept of edu star is so exciting as the bill to tell you whether they are aligned with the common core standards or whether that is just a passing line. >> one thing to think about is to think about edu star is part

of an ecosystem of tools that help to exactly what dennis said because we just thought that label on it is the same old stuff, new label. that's not what we want. and it might take some of the burden off. so this is one particular set of information coming in now, maybe quantitative information that comes from a set of task. we also want information from the people using these tools. teacher feed back come the student fee back is going to be a really important component. there is probably going to be expert reviews that will be another important component. so one of the things helpful here is to think in terms of education is multiple measures so that we begin to get the full picture of a lot of these tools. the information about how it actually help students achieve

is absolutely critical. we can't do without that, but these other pieces will be helpful and i think probably make it to some of your concern, eric, about it's not just information. it's a little bit about, wow, how do these things improve? how do they get better? how do they fit into the classroom? and i think i can give a more full picture. >> one quick thing about data. i think boland said we have so much data in education. a lot of it is manual, paper-based, hard to get what happens in the classroom and understand much more. but what we have is removed to a digital environment is a deluge of big data that's going to be the rocket goes, what can power because we will understand much more about which students in which contacts in which situation, but some are so

disaggregated the data and understand more about specifics. so i think that is definitely one of the important things and the data will also help us understand more about how people learn in general. i'll be able to understand about how people tend to learn fractions and test them in a much more rapid, much more rapid format in our previous survey manual situation. >> there seems to be some agreement that there's at least a kernel of a good idea here. i wonder if you could talk about -- but i just don't usually cost $5 million or less. so how exactly would this work? >> when they think about a budget for a hypothetical organization, we do it all architects to come up a blueprint together to develop what would look like. so we think about the statute made for an operation where the

money would come from. because vendors in the papers. or thinking about systat members, a director of research and work to collect data and also a director of outreach and interfaces in schools because we really want to not lose the voice of the teacher, both in the creation of the platform committee user ratings but was talking about desecrated just about ideas about how to teach the concept. so we're hoping that direction would interact with the school system and help aggregate the demand karen was talking about. you need a person to work with entrepreneurs and innovators. we think you'll find this proposition attractive. there's countries are looking for markets to sell them. they may have great ideas, but they don't have the sales force to go insult to a bunch schools, certainly not the 100,000 schools across the country and 15,000 school districts. so we think with so much engaged with them, the administrative leader is someone to do regulatory work.

one issue that hasn't been discussed which we should bring up his regulatory issues that need to be worked out. ben and i tried to think about those around privacy and parental permission forms, schools like in new york and there's ways we can try to use. so these are some of the staff members will have. the 5 million, were to come from? it goes back the funds you might have an influence on the result. we don't want that to happen. looking for a few forces, what is the foundation. several represented here and also, not to look to directly in your direction there, but also i looked directly at you. there's lots of different funding sources -- [inaudible] [laughter] >> -- that's how you get started on the scale. down the road we want to build something sustainable. so we think about user fees that

would be scaled given the size of the company. eventually the two be sustainable without governmental foundation. recited up the the original 5 million. >> the concept is really to leverage all the great ideas out there. i mean, the reason is it's relatively inexpensive and maybe reserpine is incorrect to some degree, but it's certainly going to be relatively inexpensive because it's about setting up a system of rules. the real innovation is sent edu star it is not going what the writing is there is. it's about creating a system where great teachers and great ideas and other entrepreneurs can come together and say okay, let's jump on this platform and try and see what works and communicate that to the public at large. when you look at how the innovation systems, it is not about knowing at the center with the right answers are. it's about creating a system of straws on the immense creativity of the different expertise and

perspectives of which is going to be right, some which will be wrong. we have so much of that in this country, so much in our school systems already. how do we engage the emoticon chevrons benefit? busway we hope that leverages somewhat. >> compared -- would they be willing to pay the fee to get evaluated? >> one dollar. [laughter] >> what is interesting in tandem with there being something like eddie's star available is also just one of the problems is also the way the industry is structured right now, where you have traditionally a situation where much bryner comes up with a product and then realizes that for that product to be profitable, they need to find that one decision-maker who is going to sign that check and pay for it for an entire district. and this is why it's very hard

for startups to get into this space because it requires this unbelievable salesforce that is not only by the dispersed, but happens to a political connection so you know so-and-so's uncle knows this person tonight at the meeting in a big fan they need their whole district. so that is problematic because it leads to all sort of distortions about what works and what doesn't. it also leads to a situation were on schwerner is not spending their time focused on users. right now the incentive is around how to iterate around that decision-maker who will sign that check instead of how to iterate around the teacher who's going to use this as a tool? the parent who uses this as as the tools on not spent jump into the district procurement groups, but actually spent trying to win over the hearts and minds of the users who should feel like this is a beautiful tool that just makes life easier for them and

makes them are effective. and so, it's also interesting because you're starting to see at least one example of this for the premium model is starting to be used. so learnzillion is for a period appeared set to context to give feedback, but really we want as many teachers and parents using for free so we can start to collect data behind the scenes and i run start to use google analytics, other tools to just find out what do folks want? what did they spend their time doing civic and double down on those things and give them what they want. and i think that is new. it is not proven yet it is definitely a lot of skeptics around this idea for your server something free, try to get a lot of folks to solve a problem and figure out how to save on premium services. it is a new model, but it's actually very exciting because i

think this goes back to the last panel, where the idea of respect for teachers often not the teachers are left out of these decisions than they are the ones using these tools have been there not the ones making decisions about which help them, it's problematic. so there's an exciting opportunity to go straight to teachers, give them something powerful and then work on making it is useful for them as possible and then go to the district leader in say 20, 30% of teachers love this. here's what we can do to sort of hold this up in a really compelling way for you so that it's not the enterprise fail. it's a bottom-up approach. that working in collaboration with a edu star type platform is really exciting in terms of the trend towards creating some and that's really helpful for teachers, parents and students. >> is almost like if we could do all these things, the other

things i would say iran that is one dimension is this notion of perturb reform. i know that's really exciting, but we've been thinking a lot about all the different things that will actually help people be smarter about what they are purchasing and also help -- we do things like aggregated purchasing. so somebody find some thing, they found a successful, how do we leverage the distribution power of the internet to share that up rather late so other people can jump on. so aggregated purchasing this one. different states and districts spend a lot of time fine-tuning and howling and we've been looking at this model, creating almost in an online wiki so people can make them better and better and that brings to a third point something called advanced market commitments are advanced market requirements. if a dozen other superintendence

to design what they would like to malika someone could build this, this is what we would really like to get better and better at articulating needs at articulating what they would like. that was actually provide the archer pernod or so were all over the place looking for great ideas. i taught to folks most potentially do have ideas or look for ideas or whatever, people who want to build something. so this information can get provided back. again, the whole ecosystem of smart supplies the demand is going to be approved. >> one of the road vintages -- about what you said about using part visionaries. i'm glad i didn't have to say that. the representative of over 3 million people who work there. what is really exciting to me is the ability when you say you don't have to find a person in the district is that when you

look at every form in this country, weatherby changes in technology and how we use that, the description i would use this way about these campfires have excellent all across the country. and by definition the campfire is not supposed to spread. it stays right here. and what we need is a brush fire. what you're describing is a system that connects people who have similar ideas or similar challenges in the district. maybe it's the district but all of a sudden it's a real influx of english language learners. and there may be only two of their schools that have never had this before. what do we do? well, it's a way of connecting to good practice with effective tools, weatherby curricula mark pryor has come a professional does to people who have liked goblins or who are like-minded and i think that is a huge possibility. there was talk about how do we care that? how to remove the doubt from

little places? i think there's a real possibility in creating circumstance we can go around the boundaries that currently stop us. >> i'm going to give you a chance to be the last person on the panel before returning to the audience if you have some do you want to add. >> sure, there's so much to say here. i do want to focus a little bit before we just are overly optimistic on one of the pieces that i thought you were really good your paper, where you talk about the types of things that this would be good for right now. and you are really clear your paper and i think in your presentation that this type of tool, at least right now, it's good for the kind of point things that are sort of almost like the app. it would be great for the apps type of situation. 83-year-old rodenticide throughout, so i need your vote. but in other words, the sort of

single implementation of the piece, the romar group uses when you talk about the macro skills for the small pieces. so i think that could add a lot of value. it does for now -- there's all sorts of products like if we think more broadly in all sorts of interventions that are more complex than that. so i think this is a bigger problem. this is a bigger piece of it in a bigger problem in need of a lot of folks working on a private letter on the different ways to slice it. and so, while i hate to go cold water, just one has to be realistic about the challenge here because it is a huge challenges though, don't walk out and say great, those people have assault, check. had remixer the innovation moves forward and does good is something we all should wrestle with for a while. thank you. so it's time for a couple

questions from the floor. i think overhears a particularly excited person. [laughter] [inaudible] >> at like to talk a little about basically my job is to help companies better somewhere between zero and 30 million graves capital -- to raise capital or reacquire. and one of the things that i would do in enforcing nodded moving forward. and i would have to say that i don't think the problem we face is fundamentally one of a lack of innovators. or for that matter, a lack of

programs that have been proven to be effective in clinical trials or otherwise. the problem that i see is that for all intents and purposes, program efficacy is a very small component of school districts purchasing decisions. you know, one of the other things i do to help my clients as i look at literally -- i have a staff that literally reports to me on every single school improvement related irs p. that comes up from the local education agencies, state education agencies in the federal government. and i look at those rfps and program efficacy is a relatively low component of the decision across the board. and i think if you do not have a

demand, that program efficacy be a component, a serious sort of entry component of the procurement decision-making process, all of the high quality evaluation in the world aren't going to make a whole lot of an act. and i think that is something you really have to address is a much broader question. >> it looks like you want to take it on. >> yes, so first of all yes, we need to raise the bar and began to ask for more information about the kinds of things for purchasing. so absolutely right on. the problem that we have and it's very difficult to get this information, especially that technology because the rapidly changing. our critical trusted one thing to do and by the time they're done, by the time they are done, the product is likely blakely changed, different archon,

right? so this is an attempt to see if we can get the economists are better information so that we can in fact begin to ask for more evidence. i think all of us above our evidence about the things that were, don't work, promise certain situations and contexts for specific students. that is absolutely an aspirational: we should build that and we need more information. as everyone sat, this is one element in a bigger ecosystem is trying to get better information. >> before that financier, can you say that again. you want to hear from economists? >> i do. i've come to respect the economists. >> it's a rare day. we should probably just go home and take a nap and celebrate. >> i introduce myself as an

economist. people are searching my wife and say, what you do? [laughter] in addition to what karen said, put yourself in the buyer shoes. if you don't know what works, how can i cacique become a meaningful part -- if he wanted to choose between seven choices and i don't know which is better than someone will, i will choose in some other bases. no metaphor is perfect, but we have hospitals and insurance companies, different systems, notoriously balkanize. we have talked yours was very different ideas innately about how to treat their parents and have lost the debate. the need for clinical trials for most pharmaceutical companies are just expensive. and yet, we see 100 times the r&d we as we do in k-12 education and turn because if they do prove their medicine is better, then guess what?

even though it's balkanize adapters don't violate a great commander is a demand created and people adopted it so i take your point. there's challenges for the buyer. there's other considerations of course, but i think if we provide information, we'll only look to the right direction, whether we get the brush fire or run against other fire retardant to remain to be seen when they have to come up as well. that the information i'm not sure i would just appoint her as a meaningful or useful part of the decision-making process. >> over here, the woman in red. >> hi, i'm a development consultant working with small business and small nonprofits. two questions, one with regard to the 501-c3 itself could you have proposed with regard to the 10 year of your board as well as

your advisory whole, especially with regard to it your membership kit to activities, investments and anything associated with the products is going to prevent them to do whatever it is they do. secondly, what our conversations you're having with the patent trademark office, small business administration, associations and even the date using the guide star as a tool and the participation of special entrepreneurs as well as aussie. >> so these are great questions. first of all, this is a proposal to the organization. a lot of the organizations are to be done rather than have been done. i started the second point, the small business administration seems like actual workers. those who are working, maybe got a loan or on software's to the platform or the patent office.

a lot of issues around patent and software they would need to learn about and i think that is what you're alluding to, right? these are all things on my mind, but they kind of get going and conversations will be had. i can't say we have a plan to deal with all that yet. the first thing with tenure for the board is an important concern to have the right kinds of violence to make sure there's not basic conflicts of interest. that is the issue dennis was raising. if the organization has credibility and people have results, there's no way we can build have results, there's no way we can build have results, there's no way we can build into this educational software space. but as their main concern. in terms of the rules, there's lots of ways to accomplish that. right now we consider options on the guidelines for the board. so that's the answer at this point. >> can i just jump in here as well? one future proposal that is an incredibly impressed by is the safeguards are trying to put up against the influence from the

companies who would like to be evaluated. there's a world littered with examples of dirt ready certification organizations to get corrupted by the conflict of interest and i think it is admirable how serious they been about that. okay, i think there's another question here. [inaudible] >> i think where the microphone, sir. >> that is twice a week i get this inquiry, where is their consumer reports for the technology we know we're going to have to bind the next couple years? said thank you for this proposal is moving forward to decide the question me to be the double firehose impact. your chart showed a process. won't there be a fire hose at both ends? one is all the technology, even as narrowly focused as karen rightly points out, but you seek to make it, but all the technology that is fair, ready to be evaluated.

the other firehose is all the folks seeking immediate reaction as 15,000 superintendent teachers and instructors have to make very important decisions over the next year and half to two years about how to implement the common core. so firehoses on each and peer-to-peer and private sector, that's great news. they build up demand immediately. >> were their excess demand for the platform of account of their lexus has been challenged as opposed to a total failure. i think it's really hard to know how large-scale the test site needs to be. the very large school districts have lots of students. it doesn't take too many large school districts to be allowed to have a sufficiently sad side to do this rapidly. the worst cases suggesting there might be a few away afterward to overcome to overcome that. hopefully we will find systems